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Summary 

Under No Child Left Behind (NCLB), state education agencies are required to 
assume new roles and responsibilities. Among these is the establishment of 
a state system to support schools identified for improvement under the Act. 
Conceptualizing and operationalizing these systems of support has been a 
challenge for many state agencies, in part because this role is a departure 
from the traditional compliance monitoring activities with which state of- 
ficials are most familiar. This brief presents data from a national survey of 
state administrators to describe trends in the implementation of this NCLB 
mandate. We begin by outlining the common components of state systems 
of support under NCLB and then suggest a set of research-supported indica- 
tors of the quality of those supports. Our purpose is to provide state officials 
and policy analysts a framework with which to assess and refine current and 
planned systems of state support. This brief is a companion to Help Wanted: 
State Capacity for School Improvement. 

State Systems of Support 
Under NCLB: Design 
Components and Quality 
Considerations 



Introduction 

State education agencies are actively 
seeking to improve their low-performing 
schools. The recent focus on "turnaround 
schools" and "dropout factories" has led 
to a lively policy dialogue about the very 
lowest-performing schools (see Calkins, 
Guenther, Belfiore, and Lash, 2007), but 
state officials are also concerned with a 
broader range of low-performing schools, 
including those on the cusp of failure. The 
current level of state activity is driven not 



only by NCLB but also by state accountabil- 
ity laws and state economic, political, and 
equity concerns. Whatever the impetus 
that has motivated state officials, national 
data indicate that all states have imple- 
mented some sort of system to support 
low-performing schools, and most states 
are seeking to refine their supports — par- 
ticularly when confronted with chronically 
failing schools (Le Floch, Martinez, O'Day, 
Stecher, Taylor, & Cook, 2007; Rhim, Hassel, 
& Redding, 2008). 





In the past, many policy efforts have fo- 
cused on state content standards, assess- 
ments, performance targets, and public 
reporting of school-level accountability re- 
sults. However, most schools that are iden- 
tified for improvement 
need more than goals 
and public pressure to 
improve — they need 

outside assistance to di- 
agnose problems, iden- 
tify solutions, and build 
internal capacity (Finni- 
gan & O'Day, 2003). 

At present, all states 
have designed and im- 
plemented supports and 
interventions to help 
schools identified for 
improvement. However, 
state officials cannot 
be sure if they have ad- 
equate capacity to assist 
increasing numbers of 
these schools, whether 
accountability strategies 
are making in-roads in 
the most challenging 
schools, or whether hard-won improve- 
ments can be sustained. In short, they can- 
not be sure if the support provided is of 
adequate quality to stimulate and main- 
tain the desired achievement gains. In the 
next few years, we are likely to witness 
further change — if not overhaul — among 
state systems of support as states work 
to address these challenges by expanding 
and/or rethinking their provision of sup- 
port. To inform this revision process, it may 
help to consider the features and quality 
of the systems of support currently operat- 
ing throughout the nation. 

In this research brief, we address two 
central questions in the design of a state 



State officials 
cannot be sure if 
they have adequate 
capacity to 
assist increasing 
numbers of schools 
identified for 
improvment, whether 
accountability 
strategies are 
making in-roads 
in the most 
challenging schools , 
or whether hard-won 
improvements can 
be sustained. 



system of support. First, what are the pri- 
mary components of state support to low- 
performing schools? Second, what are key 
indicators of the quality of supports pro- 
vided to schools? In response to the first 
question, we will map the terrain of current 
state support systems so that state officials 
can consider the array of choices open to 
them and potentially identify gaps in their 
own approaches. In response to the second 
question, we will offer a set of indicators 
(supported by prior research) that state offi- 
cials can use to evaluate the quality of their 
supports. While we do not offer a roadmap 
for improving state systems of support, 
our intent is to inform the on-going work 
of state education officials, policy analysts, 
and technical assistance providers by stimu- 
lating dialogue, system refinements, and 
evaluations of effectiveness. 



To address these questions, we present na- 
tional data on features of states' systems 
of support. In February, 2008, staff from 
the American Institutes for Research (AIR) 
surveyed state officials in all 50 states on 
their capacity and approaches to provid- 
ing ongoing support to schools identified 
for improvement. This online survey, which 
attained a response rate of 100 percent, in- 
cluded questions on the structure of state 
systems of support, states' capacity to sup- 
port school improvement, the types of staff 
that states employ within their systems of 
support, and the functions staff members 
perform to facilitate school improvement. 
Respondents were state officials with pri- 
mary responsibility for their state system of 
support for schools and districts identified 
for improvement. The survey also included 
several opportunities for state officials to 
provide open-ended responses. All quota- 
tions in this research brief, unless otherwise 
cited, are compiled from state officials' re- 
sponses to this survey. 
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Accountability Context 

In 2001, the No Child Left Behind Act took 
aim at the nation's persistent achievement 
gap by asserting that all students must 
demonstrate grade-level proficiency — as 
defined by states — in math and reading by 
the year 2014. In service of this ambitious 
goal, the law called upon states to institute 
stronger school accountability measures 
that would identify under-performing 
schools and monitor their progress toward 
meeting specified growth targets. To ensure 
that schools had the capacity to achieve 
their growth targets. Section 1117(a)(1) of 
NCLB instructed states to "establish a state- 
wide system of intensive and sustained 
support" that would provide assistance to 
under-performing schools and their corre- 
sponding local education agencies in order 
to facilitate academic improvement. 

Federal laws (NCLB, and before that, the 
Improving America's Schools Act), have 
been major stimuli in the development 
of state systems of support. Many states- 
among these California, North Carolina, 
Kentucky, Louisiana, and Massachusetts- 
had established mechanisms and programs 
to support low-performing schools well 
before NCLB made statewide strategies 
mandatory. Although these state supports 
still exist, and in many cases are more firm- 
ly established, the data presented in this 



research brief relate to support systems 
established in response to the mandates 
of NCLB. In surveying states, we chose to 
focus on NCLB systems of support because 
they are a common requirement of all 
states and thus allow us to examine trends 
on a national scale. However, our discus- 
sion of system components and research- 
supported indicators of quality could apply 
equally to other state initiatives to support 
low-performing schools. 

Supports for school improvement should be 
understood within a broader framework 
of school accountability. A results-based 
accountability system is designed to fo- 
cus attention on student performance (by 
setting performance targets), to motivate 
educators to change (by public disclosure 
of performance and through rewards and 
sanctions), and to build capacity (through 
external assistance and resources) (O'Day 
and Bitter, 2003). A summary overview of 
the NCLB accountability process is depicted 
in Exhibit 1. 

The focus of this research brief is just one 
component of this broader framework: 
provide appropriate support to targeted 
schools (the bolded box, below). For this 
reason, we will not discuss other important 
elements of the accountability system, in- 
cluding possible rewards and sanctions. 



Exhibit 1: NCLB Accountability Approach to Improving Student Achievement 
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Design of State Systems of 
Support 

When NCLB went into effect, research on ef- 
fective state support strategies was limited, 
yet states were required 
to make numerous de- 
sign decisions. States' 
designs for systems of 
support were shaped 
by their own internal 
capacity (Rhim, Hassel, 
and Redding, 2007), fed- 
eral law and guidance 
(Le Floch et al., 2007; 
Laguarda, 2003) and 
each state's assumptions 
about how to achieve 
better results (Reville, 
2007; Education Alliance 
at Brown University, 
2006). State capacity — 
which we explore more 
fully in a companion re- 
search brief 1 — includes 
financial and human resources, technolog- 
ical capacity and the political will to make 
and support required changes. Together, 
these shape the breadth, depth, and inten- 
sity with which states can respond to the 
demand for support. 



Many officials 
adhere to the 
general premise 
that failing schools 
lack some degree 
of internal capacity 
and will , but that 
with adequate 
information , external 
guidance , pressure , 
and funding , they 
should be able to 
improve. 



mation, external guidance, pressure, and 
funding, they should be able to improve. 
Other assumptions vary from state to state 
and underlie a host of decisions about how 
to most effectively design, implement, and 
sustain school improvement. Such decisions 
include how much and what types of exter- 
nal support should be provided, who should 
provide support and to whom, to what ex- 
tent and on what basis should support be 
differentiated, what should constitute the 
sequence of steps in the school improve- 
ment process, and how much authority 
schools should have in selecting improve- 
ment strategies. 



North Carolina is among the few states that 
explicitly articulate the theory of action 
that grounds its state system of support. 
For example. North Carolina specifies how 
its approach is anchored in the principles 
of building district capacity to support all 
schools, providing more intensive support 
to chronically low-performing schools, and 
tailoring support to specific district and 
school needs (for more details see North 
Carolina Department of Public Instruction, 
2007; see also Reville, 2007 for a discussion 
of how theories of action can inform the 
design of state system of support). 



Although rarely articulated, state officials 
generally share a number of assumptions 
about how school improvement takes 
place, inherited from the framework set 
out in NCLB. State officials hold a set of 
beliefs (sometimes described as a theory 
of action) about how schools work, why 
schools fail, how schools change, and how 
to help schools change effectively (see, for 
example, Argyris & Schoen, 1974; Argyris, 
2000; and Fullan, 2007). Many officials ad- 
here to the general premise that failing 
schools lack some degree of internal capac- 
ity and will, but that with adequate infor- 



Components of state 
support 

In our review of the structure of state sys- 
tems nationally, we have identified key 
components of these systems, each of 
which we discuss below. These components 
are: (1) tools to support the school improve- 
ment process, (2) providers who deliver sup- 
port, (3) support activities, (4) funding for 
school improvement, and (5) the content of 
the improvement strategies themselves (see 
Exhibit 2). 
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Exhibit 2. Components of State Systems of Support for Low-Performing Schools 






Tools to support the school im- 
provement process 

By 2008, most states had developed a com- 
prehensive framework for school improve- 
ment, that is, an outline of key components 
and/or steps that schools should consider as 
they plan their improvement strategy. As- 
sociated with these frameworks, state of- 
ficials often develop tools (templates, ru- 
brics, on-line protocols) that take schools 
through a step-by-step process of planning 
and implementation. For example, Michi- 
gan's framework identifies five "strands" 
on which low-performing schools should 
focus: (1) teaching for learning, (2) leader- 
ship, (3) personnel and professional learn- 



ing, (4) school and community relations, 
and (5) data and information manage- 
ment. For each of these strands, the state 
education agency has delineated stan- 
dards, benchmarks, and key characteristics. 
Similarly, New Mexico's "Education Plan for 
Student Success" is anchored by (1) qual- 
ity teaching and learning, (2) professional 
culture and collaborative relationships, (3) 
effective leadership, and (4) support for 
system-wide improvement. 

In states that have developed frameworks 
for school improvement, the intent is for 
these to guide each step of the school im- 
provement process, including the use of 
planning tools, activities of support per- 
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sonnel, and funding priorities. Based on 
our review of state education agency web- 
sites in early 2008, 32 states had developed 
frameworks and 43 states had developed 
tools designed to support the school im- 
provement process. 

Support providers 

Under NCLB, state systems of support are 
required to include individuals or groups 
who provide external support to identified 
schools. Support is provided by staff from 
a range of public organizations — including 
state education agencies, regional assis- 
tance centers, and school districts — as well 
as external consultants and private organi- 
zations. 

Results from AIR's 2008 survey indicated 
all states had designated SEA staff who 
were engaged in activities related to their 
system of support; in addition, all but one 
state employed other individuals or groups 
to supplement the efforts of their SEA. In 
some states, particularly those with large 
numbers of schools identified for improve- 
ment and/or limited SEA staff resources, the 
decision to add external service providers 
into their system of support resulted from 
sheer necessity, due to their lack of inter- 
nal capacity. In other states, the inclusion 
of external staff has enabled SEAs to offer 
schools more specialized or intensive levels 
of support. Only one state indicated that 
its system of support was comprised solely 
of SEA staff, and state officials emphasized 
the coherence inherent in a centralized ap- 
proach. One state official explained, "Since 
our team works out of the Department of 
Education office, we are cohesive in our 
service delivery efforts. We do coordinate 
with other state department units to de- 
liver coordinated services to schools and 
districts." 



Among states that have incorporated ad- 
ditional personnel, 46 reported using con- 
sultants in 2008, either by hiring individuals 
with specialized expertise or, in some cases, 
by contracting with external organizations 
to provide support. In Tennessee, for ex- 
ample, the "exemplary educator" program 
is administered through a private organi- 
zation, Edvantia, Inc., which selects, trains, 
and monitors the activities of exemplary 
educators. 2 A sizeable number of states (29) 
noted their support system's use of district 
staff. States generally provide training to 
the district staff operating within their sys- 
tem of support and provide some level of 
oversight and monitoring to ensure school 
needs are met. 

Half of all states (25) built their system of 
support around a regional network of ser- 
vice centers or county offices, many of which 
existed prior to NCLB. Regional systems can 
be particularly important for geographically 
large states where travel to schools can be 
a challenge for centrally-located SEA staff. 
Regional units still fall under the purview 
of the SEA but focus on schools and districts 
within their region. 

Professional networks, or other activities 
to foster collaboration among school im- 
provement professionals, appear to be an 
emergent trend in state systems of support 
(Reville, 2007). Massachusetts, for instance, 
established a School Support Specialist 
Network wherein district- and state-based 
School Support Specialists would meet 
at least once a month to share informa- 
tion, resources, and strategies for assist- 
ing low-performing schools. Massachusetts 
also convenes its urban superintendents 
on a monthly basis to facilitate profes- 
sional networking. In Michigan, principals 
from schools in need of improvement par- 
ticipate in a series of leadership institutes, 
which helps establish professional networks 
among principals in high priority schools. 
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Support activities 

Federal statute specifies some of the func- 
tions of a state system of support (see NCLB 
section 1117(a)(5)(B)) including analyzing 
data, designing a school improvement plan, 
and monitoring implementation of the 
plan. Additionally, scholars of state systems 
of support have suggested activities that 
are appropriate (and desirable) for state 
systems of support including planning and 
implementation, leadership support, use 
of data, instructional support, and profes- 
sional development (Reville, 2007). 



District and regional staff often serve many 
of the same school support functions as 
SEA staff, though the number of states 
that use them in such capacities is some- 
what low overall since fewer states incor- 
porate district and regional staff into their 
support systems (29 and 25, respectively). 
Consultants, who are active in more state 
support systems (46) than staff from district 
or regional offices, frequently are used to 
support needs assessment (in 35 states) and 
improvement planning (in 35 states) pro- 
cesses; however, only 18 states use consul- 
tants to help identify or acquire resources 
(see Exhibit 3). 



In many states, SEA staff are directly in- 
volved in performing these support func- 
tions, especially providing assistance with 
needs assessments (in 41 states), resource 
acquisition (in 36 states), and school im- 
provement planning (in 40 states). Fewer 
states (19) employ SEA staff to serve as 
leadership or instructional coaches, but a 
fair proportion of states engage consul- 
tants (32 states), district staff (21 states), 
and regional service providers (1 5 states) to 
perform these roles. 



Targeted Funding for Improvement 
Activities 

Financial resources are another critical 
ingredient of state systems of support. 
Money alone cannot solve the problems 
facing low-performing schools, but ad- 
ditional funding can often enable schools 
to make recommended changes designed 
to improve student outcomes (Mintrop & 
Trujillo, 2005; Dwyer et al., 2005; Reville et 
al., 2005; Ascher, Ikeda, & Fruchter, 1998; 
Davis, McDonald, and Lyons, 1997). NCLB 



Exhibit 3: Number of States Using Particular Types of Staff 
to Serve Various Support Functions, 2007-08 




Coordinating Analyzing data or Developing an Identifying or Acting as a 
system of support identifying needs improvement plan acquiring resources leadership or 

instructional coach 



District Staff 
Staff 
Consultants 
SEA staff 



^ Regional 

□ 



Source: AIR Survey of State Education Agency Capacity, February 2008. 

Note: Three states that did not provide data on their support activities have been excluded from this analysis. 
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Design Questions: To what degree should support be differentiated? 

States differ greatly in the degree to which they tailor support based on school characteristics. Often, such 
differences are related to capacity constraints as much as state-specific approaches to improvement. For 
example, states typically commit the most support to Title I schools, which are backed by federal funding, and 
offer either reduced or no support to non-Title I schools because less, if any, funding is available to assist them 
(see Exhibit 4). 

In the face of limited resources, several states have opted to target support to their lowest performing schools 
to ensure that their neediest schools receive assistance. By 2008, 38 states had developed tiered systems of 
support in which the form and intensity of support increased according to the severity of a school's accountabil- 
ity designation. For instance, in Texas, Title I schools identified for improvement receive support from a techni- 
cal assistance provider (or TAP). The number of days that a TAP is required to spend on the school site varies 
both by school size and by the number of years a school has been identified for improvement: as few as 20 days 
for a small school that just entered school improvement, but as many as 45 days for a large school that reaches 
restructuring status. States may also consider indicators other than school improvement status, such as school 
achievement levels or reasons for failing AYP. 

SEA capacity is not the only driver of support differentiation. State officials may also have assumptions about 
the extent to which successful school improvement is related to school context. In 2008, many states indicated 
that they adjust their support strategies in response to specific school and district contexts (see Exhibit 4). 
States may, for instance, modify their provision of support based on a school’s district setting. Minnesota, 
which uses a regional system to provide support forthe majority of its school districts, has developed an alter- 
nate approach for serving schools in its two largest cities where greater concentrations of schools are identified 
for improvement. SEA staff work directly with these schools, providing assistance that is similar to what is 
offered by the state’s regional staff but customized to meet the needs of these two urban districts. States may 
also tailor their services to address school needs regarding special student populations such as students with 
limited English proficiency (LEP) or students following an individualized education plan (IEP). For example, 
Michigan is developing a targeted audit process for schools that miss their AYP targets only for the English 
language learners or special education subgroups. Frequently, states will support schools with such needs by 
assigning individuals with specialized expertise in these areas to work with these schools. 

Exhibit 4: Number of States that Differentiate Support Based on 
Various School Criteria, 2007-08 

50 




Title I Students Students Elementary Middle High Urban Rural 

Schools with LEP with an IEP Schools Schools Schools Schools Schools 



Source: AIR Survey of State Education Agency Capacity, February 2008. 

Note: Two states that did not provide data on their approach to support differentiation have been excluded from 
this analysis. 

Lastly, more than a third of all states (19) indicated that they differentiate support for low-performing schools 
according to schools’ grade levels: 17 states reported tailoring support specifically for elementary schools and 
another 17 states reported doing so for high schools while 15 states indicated that they differentiate support 
for middle schools. Recognizing that schools face different sets of challenges depending upon the grade levels 
they serve, these states have endeavored to align the support schools receive with the challenges associated 
with their particular grade spans. 
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provides some funding for school improve- 
ment through provisions detailed in sec- 
tion 1003(a) of the law. States are required 
to set aside a portion of their Title I funds 
to establish a School Improvement Fund, 
which is then disbursed to schools on ei- 
ther a competitive or formulaic basis. Be- 
cause states vary greatly in the size of their 
Title I allocation in proportion to the num- 
ber of schools identified for improvement, 
corrective action, or restructuring, the size 
of the section 1003(a) grants to schools 
varies. For example, in 2004-05, Arkansas 
had $2.1 million for 272 schools identified 
for improvement, while Minnesota had 
$2.3 million in school improvement funds 
for 38 schools identified for improvement 
(McClure, 2005). Federal funds are not the 
only source of financial support for low- 
performing schools. States also provide 
targeted state funds to support school im- 
provement (often through targeted grant 
programs), and private foundations sup- 
port improvement strategies as well. 

School-based strategies to stimu- 
late improvement 

At the heart of the school improvement 
process is a set of strategies and activities 
through which stakeholders hope to stimu- 
late school-level changes that will lead to 
improved student outcomes. As described 
earlier, state-level supports in the form of 
tools, funding, and external service provid- 
ers are generally geared toward helping 
schools select appropriate strategies in re- 
sponse to their identified needs and toward 
empowering schools to carry out those in- 
terventions to generate improvement. 

School improvement strategies vary in 
scope from those that encompass numer- 
ous areas of a school's operation to those 
that hone in on particular school activities 
or content areas. At the most comprehen- 



sive end of the spectrum are whole school 
reform models such as 
America's Choice, Success 
for All, Talent Develop- 
ment, and High Schools 
that Work, which offer 
a cohesive package of 
what are designed to 
be mutually reinforc- 
ing practices to increase 
student achievement. 

Collectively, the behav- 
iors prescribed by these 
models address many 
different school func- 
tions, including instruc- 
tion, scheduling, organi- 
zational structures, and 
staff development, among 

More focused programmatic interventions 
tend to concentrate on particular school 
activities (such as parent involvement or 
teacher collaboration) or content areas 
(often reading or math), though some 
of these targeted interventions may cut 
across multiple aspects of the school. For 
example, a school might 
alter its instructional 
schedule to create com- 
mon teacher planning 
time as a means of fos- 
tering the development 
of a professional learn- 
ing community, facili- 
tating interdisciplinary 
approaches to instruc- 
tion, and/or improving 
school climate (Morrisey, 

2000; Maclver, 1990; 

Warren & Muth, 1995). 

Another popular school 
improvement strategy is the organization 
of classroom "data walks" through which 
school leaders conduct regular teacher ob- 
servations to collect and analyze data on 



A key question 
influencing states' 
approaches to 
providing support is 
how much authority 
schools should have 
in determining which 
interventions they 
will implement. 



At the heart of the 
school improvement 
process is a set 
of strategies and 
activities through 
which stakeholders 
hope to stimulate 
school-level changes 
that will lead to 
improved student 
outcomes. 



others. 
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Design Question: Who should decide which improvement strategies are appropriate? 

A key question influencing states’ approaches to providing support is how much authority schools should have 
in determining which interventions they will implement. Answers to this question, which reflect important as- 
sumptions about the relative capacities of schools and their external stakeholders, shape not only the types of 
support states choose to provide but also how these chosen supports interact with schools. 

Some states endorse a view that school-level stakeholders (generally with some external assistance) are in the 
best position to understand their own context and challenges. In these cases, school staff themselves decide 
which solutions are appropriate, often with some level of facilitation from the state. The assumption is that hav- 
ing schools develop their own approaches to tackling their most salient problems will more naturally encourage 
school-level buy-in, implementation, and sustainability. It also supposes that schools have some basic level of 
internal capacity, and just need a little help to articulate, refine, and implement solutions. Accordingly, states 
subscribing to this approach might focus their support structures on helping schools make informed choices, 
for example by coordinating a school-based planning process or providing information or research on promising 
improvement strategies. Such states might position their support providers to serve as critical friends, process 
facilitators, and/or information resources rather than decision-makers or advocates of particular strategies. 

In Washington state, which explicitly empowers school-based staff to choose which interventions to implement, 
teams of school staff engage in a year-long process of analyzing data, identifying goals, and proposing strate- 
gies. To support these school teams in composing a set of interventions that will best address their needs, the 
state hosts conferences that introduce research-based strategies (such as Professional Learning Communities 
or Data Walks), allocates funding to pay for school staff’s improvement planning time, provides tools to guide 
the improvement planning process, and assigns a school improvement facilitator to assist in the development 
of a school improvement plan. 

Other states opt to exert greater influence over the selection of underperforming schools’ improvement strate- 
gies. They may require or endorse the use of particular interventions, either by establishing state-level policies 
to that effect or by authorizing external service providers to select interventions for schools. The most frequent 
scenario where states constrain schools’ decision-making authority is one in which authority is progressively 
retracted from the school as their accountability designation becomes more severe. For example, schools 
identified for improvement might retain a good deal of decision-making authority, working closely with external 
support staff (perhaps a leadership coach ora school improvement team) that guide the school but do not 
impose solutions. If a school enters into corrective action or restructuring, the district, a regional organization, 
or an external governing board becomes the decision-maker. In such cases, state officials rely on the expertise 
of these organizations to select an appropriate intervention and do not second guess their decisions. Such 
policies reflect assumptions about the level of internal capacity in failing schools (i.e., this capacity is increas- 
ingly in doubt as school failure persists) as well as assumptions about the appropriate level of state control. 

In Arkansas, schools that reach the corrective action phase of school improvement are strongly encouraged 
to implement the strategies prescribed in the America’s Choice school reform model, unless the schools can 
demonstrate that they have the capacity to substantially improve outcomes across all student groups under 
their existing set of improvement strategies. Somewhat less prescriptive is Hawaii, which in 2005 identified 
three external organizations (America’s Choice, ETS Pulliam, and Edison Schools) to work with schools at the 
restructuring level. 

Instances in which states enter into exclusive contracts with providers of externally-developed interventions 
within the context of their system of support are relatively rare. However, reliance on external interventions- 
ideally backed by scientifically-based research-is a feature of all state systems of support, to varying degrees. 
In states that choose to emphasize the utility of externally-developed strategies, the premise is that low- 
performing schools lack the capacity to improve by themselves and will best be served by external organizations 
that bring knowledge, on-going research, professional development and other supports. Developer organiza- 
tions have generally mapped out the steps for implementing their strategies (and may provide associated 
supports, fora fee). Moreover, there is often (but not always) evidence of the effectiveness of these interven- 
tions through third-party research reviews such as those provided by the What Works Clearinghouse or directly 
by the intervention developer. For these reasons, externally-developed strategies may seem like a safer bet for 
schools; however, locally-developed or “homegrown” strategies may also be successful in stimulating improve- 
ment, depending on the scale of the challenge and the capacity and resourcefulness of school stakeholders 
(Simmons, 2006). 
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pedagogy and instruction (Bloom, 2007; 
Downey, Steffy, English, Frase, & Poston, 
2004). Other interventions may address the 
use of instructional time (either extending 
or re-organizing the use of time) or involve 
changes to the internal organization of 
the school (creating grade clusters or small 
learning communities). Because program- 
matic interventions tend to be somewhat 
limited in scope, schools typically rely on an 
array of such strategies to provide a more 
comprehensive approach to their improve- 
ment needs. 

Once schools have conducted a needs as- 
sessment, identified weaknesses, and se- 
lected improvement strategies, they must 
implement the strategies. Relatively few 
states provide support throughout the 
implementation process. One example of 
a state that does provide implementation 
support is Delaware. Through the Dela- 
ware Educational Support System (DESS), 
school improvement facilitators provide 
support for districts with schools identi- 
fied for improvement. These facilitators 
work with districts as long as the schools 
are identified. An important component of 
their work is to facilitate school-to-school 
relationships that promote implementa- 
tion of common strategies. 

Indicators of Quality of 
State Supports 

Research has highlighted several character- 
istics of external assistance that appear to 
be related to change in practice. To date, 
however, there is little research on the actu- 
al quality of the support provided through 
state systems, and few studies have at- 
tempted to link state supports with student 
achievement effects. 3 To advance policy dis- 
cussions, we offer a possible framework of 
indicators of the quality of state supports. 
We identify eight indicators of quality that 



policymakers and state officials should con- 
sider both in the design and implementa- 
tion of state systems of support. These 
build on the framework of policy attri- 
butes developed by Andrew Porter and col- 
leagues (1988) as well as others who have 
used and modified this framework (for ex- 
ample, Finnigan & O'Day, 2003; Desimone, 
2002). In addition, we have reviewed stud- 
ies of external support to schools, including 
support from both states and districts, to 
identify additional attributes that scholars 
cite as strengths (or, conversely, attributes 
whose absence scholars cite as shortcom- 
ings) in contributing to effective school im- 
provement efforts. 

When considering quality, there are two 
sides of a state system of support: the sys- 
temic features (that is, the set of policies 
and structures that compose the system of 
support), as well as the actual support that 
is provided to school-level staff. The first 
four indicators listed below may be used 
to gauge the quality of systemic features 
as well as local support. 

The latter four indicators 
are primarily measures 
of the support received 
at the school level. Al- 
though states may set 
policies that enhance or 
constrain the latter four 
measures, they are most 
relevant in the context 
of school needs. These 
eight indicators include: 

♦ Coherence: There 
is a need for co- 
herence within a system of support, 
as well as among the supports of- 
fered to low-performing schools 
(Lane, 2007; Reville, 2007; Education 
Alliance at Brown University, 2006; 
Finnigan & O'Day, 2003; Porter, 1994). 



We identify eight 
indicators of quality 
that policymakers 
and state officials 
should consider 
both in the design 
and implementation 
of state systems of 
support 
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State policies should collectively rein- 
force and not contradict one anoth- 
er. Various state approaches should 
not create unnecessarily duplication 
of efforts, work at cross purposes, or 
confuse school staff. For example, if 
states have multiple support initia- 
tives (perhaps a high school improve- 
ment initiative and an NCLB system 
of support) the various support pro- 
viders should be encouraged to com- 
municate and coordinate efforts for 
specific schools. 

♦ Comprehensiveness: Although an in- 
dividual school may only need to tar- 
get one type of challenge, the state 
system as a whole should be compre- 
hensive; that is, the system should be 
designed to address the wide range 
of variables that can limit student 
learning (Rennie Center, 2004; Wein- 
baum, 2005). School challenges are 
often multi-faceted. For example, 
students may exhibit specific academ- 
ic weaknesses that may be associated 
with a lack of alignment between in- 
struction and standards, inadequate 
professional development, a lack of 
community support, or ineffective 
school leadership. 

♦ Stability: State systems should be 
flexible enough to adapt to feedback 
from the field but yet feature a stable 
core of supports and strategies (Por- 
ter, etal., 1988; Porter, 1994; Finnigan 
& O'Day, 2003). If stakeholders are 
tempted to conclude that financial 
supports will be discontinued, the 
individuals providing support are not 
committed to the process. Likewise, 
if political will is lacking, they will 
be less likely to buy into the change 
process with the degree of commit- 
ment that puts long-term success 
within reach. In addition, excessive 
turnover or shuffling among support 



providers at the local level can pose 
challenges as schools are frequently 
forced to adjust to providers' varying 
personalities, recommendations, and 
expertise (David, Kannapel, & McDi- 
armid, 2000). 

♦ Responsiveness: If state officials 

monitor their support system, pay at- 
tention to the feedback from stake- 
holders, and subsequently refine their 
policies, this will ultimately strength- 
en the system as a whole (Le Floch & 
Boyle, 2005). In contrast, systems that 
are unresponsive to emergent chal- 
lenges will stagnate and lose credibil- 
ity among stakeholders. 

♦ Intensity: The intensity of support — 
both in terms of the number of days 
of assistance or the absolute dollar 
amount of grants — is an important 
feature influencing the extent to 
which state-provided supports can 
foster and sustain school-level change. 
As such, questions regarding intensity 
are central policy considerations for 
all states (Rhim, Flassel, & Redding, 
2007; Reville, 2007; Education Alliance 
at Brown University, 2006; Mintrop 
& Trujillo, 2005; Finnigan & O'Day, 
2003; Laguarda, 2003; Davis, McDon- 
ald, & Lyons, 1997). If resources are 
spread too thin, the support will have 
limited impact on school practices 
and student outcomes. On the other 
hand, highly intensive support may 
stimulate change but can reach fewer 
schools. States vary greatly in terms 
of the intensity of support provided, 
from one-shot meetings to nearly 
full-time support staff assigned to tar- 
geted schools. There is little evidence, 
however, on the factors that might 
determine the right level of support 
in a particular context or on the strat- 
egy states can use to attain an appro- 
priate balance between high-intensity 
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support and reaching the maximum 
number of schools. 

♦ Prescriptiveness: Although there is 
there is no "right" level of prescrip- 
tiveness appropriate for all contexts, 
the degree to which choices of school- 
level reform approaches should be 
constrained or interventions speci- 
fied is a key consideration as states 
endeavor to provide high-quality 
support (Porter, et al., 1988; Porter, 
1994; Finnigan & O'Day, 2003). Clear- 
ly, there are undesirable extremes: 
guidance so vague that it provides no 
information of value, or minutely de- 
tailed protocols that are burdensome 
and fail to acknowledge educators 
as professionals or the local context. 
As with intensity, state officials must 
seek the appropriate balance. 

♦ Fit: The "fit" of support encompasses 
many features, including the align- 
ment of the expertise of a support 
provider to a specific school's needs 
and the fit between a school's chal- 
lenges and the selected intervention 
(Rennie Center, 2005; David, Kan- 



napel, & McDiarmid, 2000; Ascher, 
Ikeda, & Fruchter, 1998). If there is 
a mismatch — for example, a school 
improvement facilitator with subur- 
ban experience is assigned to a rural 
school — it may be more difficult to 
foster meaningful dialogue, to iden- 
tify appropriate interventions, to im- 
plement with fidelity, and to sustain 
improvement strategies. 

♦ Timeliness: Many steps in the school 
improvement process fit within a se- 
quence of activities that is constrained 
by the school year itself. Delays with 
the provision of resources — whether 
financial or human — can limit the 
capacity of a school to undertake im- 
provement strategies. For example, 
if a school improvement facilitator 
is not assigned to a low-performing 
school until February, the school staff 
will have lost over half the school year 
during which productive activities 
could have been initiated. If funds 
are not disbursed in a timely man- 
ner, then school leaders may need to 
scramble to cover budget shortfalls. 



Additional Design Questions 

• Has the state articulated the principles and priorities that drive the state system of support? Do compo- 
nents of the system align with those priorities? 

• How do state improvement initiatives complement (or interfere) with each other? Do schools receive sup- 
port from multiple organizations, and if so, are there requirements that they communicate and collaborate 
with each other? 

• Based on state capacity, how much time should support providers spend on-site? Should the state consider 
increasing levels of low-intensity supports, or decreasing high-intensity supports? 

• Does the state provide adequate information to schools on research-based strategies? How can the state 
ensure that support staff are make informed and strategic decisions about interventions? 

• Does the state have mechanisms in place to ensure that support providers are well matched to the schools 
in which they work? 

• Does the state have appropriate expertise to address different challenges schools face? How can the state 
leverage additional expertise? 

• Can the state remove any barriers that might inhibit the timeliness of support? 

• How long do schools work with a given support provider? How can the state ensure that support providers 
are not withdrawn at critical junctures? 
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or will hold off on expenditures for 
key resources. 

Conclusion 

Although rubrics to evaluate statewide 
systems of support are still rather nascent 
in their development, scholars as well as 
school- and district-level stakeholders have 
underscored the importance of states' ef- 
forts to regularly assess their provision of 
support, to foster ongoing and meaningful 
feedback loops with stakeholders across 
the system, and to carefully hone their de- 
livery of support in response to school and 
district needs (Rhim, Hassel, & Redding, 
2008; Kerins, Hanes, and Perlman, 2008). 
We hope that the indicators of quality 
above — and our discussion of components 
of state systems of support — will be used 
to advance dialogue, design, and evalua- 
tion of state systems of support. Evaluators 
may opt to include these elements when 
developing interview protocols, evalua- 
tive rubrics, or survey scales. The quality 
indicators may also serve as a lens through 
which state officials can consider informal 
feedback from stakeholders, or quick- 
turnaround data collections that provide a 
snapshot of support services. In the coming 
years, additional schools across the coun- 
try will have to begin the hard work of im- 
provement in the coming years. We have 
proposed a purposefully straightforward 
framework in the hopes that it will be use- 
ful to the policy community as it designs 
and implement ever more effective state 
systems of school support. 



Endnotes 

^ee “Help Wanted: State Capacity for School Improvement” 
available at www.air.org. 

2 Note that when states contract with consultants or private 
organizations to supplement the activities of SEA staff, their 
roles are substantively different from those of developers of 
whole-school reform models, curricular programs, or other 
vendors of school-based interventions. Indeed, some states 
have established ethical guidelines prohibiting consultants from 
proposing school strategies in which they have a financial stake. 

3 There are some exceptions: some scholars have studied the 
relationship between state systems of support and student 
achievement. For example, O’Day and Bitter (2003) reported 
that the contribution of California’s Immediate Intervention/ 
Underperforming Schools Program to improved student achieve- 
ment was minimal. Likewise, Huberman, Dunn, Stapleton, and 
Parrish (2008) analyzed student achievement in schools that 
received assistance through Arizona's system support. 
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